Supervised Nonnegative Tensor Factorization with Maximum-Margin Constraint

Non-negative tensor factorization (NTF) has attracted great attention in the machine learning community. In this paper, we extend traditional non-negative tensor factorization into a supervised discriminative decomposition, referred as Supervised Non-negative Tensor Factorization with Maximum-Margin Constraint (SNTFM). SNTFM formulates the optimal discriminative factorization of non-negative tensorial data as a coupled least-squares optimization problem via a maximum-margin method. As a result, SNTFM not only faithfully approximates the tensorial data by additive combinations of the basis, but also obtains a strong generalization power to discriminative analysis (in particular for classification in this paper). The experimental results show the superiority of our proposed model over state-of-the-art techniques on both toy and real world data sets. Introduction In many real-world applications, data intrinsically come in the form of tensors, or multi-dimensional arrays. For example, a gray image can be represented spontaneously as a 2-nd order tensor. In the last decade, decompositions and low-rank approximations of tensors have been studied extensively in ample fields, including computer vision, bioinformatics, neuroscience, and data mining (Kolda and Bader 2009; Tao et al. 2007; Wu, Liu, and Zhuang 2009). Meanwhile, the non-negativity constraint has been proved to be indispensable and useful when dealing with non-negative data in face recognition (Wang et al. 2005), biological analysis (Kim and Park 2007), psychometric (Murakami and Kroonenberg 2003) and gait recognition (Tao et al. 2006). As a result, many researchers focused on Non-negative Tensor Factorization (NTF) in the past few years (Shashua and Hazan 2005; Kim and Park 2012; Mørup, Hansen, and Arnfred 2008; Friedlander and Hatz 2008). NTF, as a more general form of the well-studied Non-negative Matrix Factorization (NMF), aims to obtain a parts-based representation Copyright c © 2013, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. of high-dimensional data object, so that the target data can be expressed by multi-linear combination of non-negative components. In comparison with NMF, in NTF the structural information is reserved in the data, while the vectorization of the object tensor in NMF may result in information loss. However, most of algorithms proposed for NTF or NMF act as unsupervised manners that cannot exploit the inherent discriminative priors (corresponding class labels) of the data objects. This knowledge is in fact useful in many real world applications, such as images with tags etc. This paper is dedicated to developing a supervised tensorbased factorization, referred as Supervised Non-negative Tensor Factorization with Maximum-Margin constraint (SNTFM). SNTFM extends traditional non-negative tensor factorization into a supervised decomposition via a maximum-margin method (specifically a support vector machine), which is formulated by coupling the approximation of tensorial data (in terms of faithful reconstruction using additive combinations of the basis) with a maximum margin constraint (in terms of a generalized discriminative power). Maximum margin classifiers such as support vector machines (SVMs) (Cherkassky 1997) and Maximummargin Markov Networks (M3N) (Roller 2004), have been successfully applied in a wide range of classification problems. Maximum-margin methods commonly construct an optimal separating hyperplane that maximizes the margin (i.e. the distance between the hyperplane and the nearest data point of each class) by mapping the input space into an associated reproducing kernel Hilbert space. It has been shown that such methods are appealing due to the existence of strong generalizations, derived from the well-known kernel trick of the learning algorithm. (Kumar, Kotsia, and Patras 2012) proposes a method in which the acquired projections are chosen so that they maximize the discriminative ability through a maximum margin method. (Das Gupta and Xiao 2011) extends the maximum-margin NMF into a kernel (non-linear) one. However, their work are limited to the factorization of vectorized data instead of tensorial data objects. This work appreciates both the decomposition as well as Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence